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March  9,  1995 

MEMORANDUM  FOR  DIRECTOR  DEFENSE  INFORMATION  SYSTEMS 

AGENCY 

SUBJECT:  Audit  Report  on  Staffing  Requirements  at  the  Defense  Megacenters 
(Report  No.  95-140) 


We  are  providing  this  report  for  your  review  and  comments.  Comments  on  a 
draft  of  this  report  were  considered  in  preparing  the  final  report.  This  report  is  the  first 
of  two  on  the  Defense  megacenters.  TTiis  report  discusses  the  personnel  staffing 
requirements  for  all  16  Defense  Megacenters.  The  second  report  will  discuss  the 
operations  at  the  Defense  Megacenter  St.  Louis,  Missouri. 

DoD  Directive  7650.3  requires  that  all  audit  recommendations  be  resolved 
promptly.  We  revised  Recommendations  2.  and  3.  to  clarify  actions  to  be  taken. 
Therefore,  we  request  that  by  April  10,  1995,  the  Defense  Information  Systems 
Agency  provide  additional  comments  on  the  revised  recommendations. 

If  you  have  questions  on  this  audit,  please  contact  Ms.  Mary  Lu  Ugone,  Audit 
Program  Director,  at  (703)  604-9529  (DSN  664-9529)  or  Ms.  Cecelia  A.  Miggins, 
Audit  Project  Manager,  at  (703)  604-9542  (DSN  664-9542).  The  distribution  of  this 
report  is  listed  in  Appendix  E.  The  audit  members  are  listed  inside  the  back  cover. 

David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 


Office  of  the  Inspector  General,  DoD 


Report  No.  95-140 

(Project  No.  4RE-5034) 


March  9,  1995 


STAFFING  REQUIREMENTS  FOR  THE 
DEFENSE  MEGACENTERS 


EXECUTIVE  SUMMARY 


Introduction.  We  performed  the  audit  in  response  to  a  request  from  the  Insnector 
General,  Defense  Information  Systems  Agency  (DISA),  for  assistance  in  reviewing  the 
operations  of  the  Defense  megacenters.  As  the  operational  manager  of  the  Defense 
megacenters  DISA  Western  Hemisphere  (WESTHEM)  developed  the  staffing 
requirements  for  those  centralized  information  processing  facilities.  g 

2fifcCtlVe’  ?e  announC£;d  audit  objective  was  to  evaluate  the  efficiency  and 
effectiveness  of  operations  of  the  Defense  Megacenter  St.  Louis,  Missouri  This  report 
discusses  the  objechve  pursued  by  this  segment  of  the  audit-the  method  selectedP  by 
SA  Western  Hemisphere  to  determine  staffing  requirements  for  the  Defense 

meSnfnrS‘  fWealso  evaluated  internal  controls  applicable  to  the  DISA  WESTHEM 
methodology  for  determining  estimates  of  staffing  requirements.  A  subsequent  report 
will  discuss  the  operations  of  the  Defense  Megacenter  St.  Louis,  Missouri  P 

Audit  Results.  DISA  Western  Hemisphere  did  not  use  an  appropriate  methodology  to 
estimate ^the  staffing  requirements  for  the  16  Defense  megaSntL  As Tresuh  the 
projected  staffing  requirements  could  lead  to  either  overstaffing  or  understaffmg  of  the 

col^alSrfro^e^  C°St  ^  showil*  “ifplrson^ 

costs  and  benefits  from  the  Defense  megacenters  consolidation  needs  to  be  revised 
before  presentation  to  Congress  in  March  1995.  Details  are  in  Part  II. 

Internal  Controls.  We  identified  no  material  internal  control  weaknesses  relating  tr> 
the  methodology  used  by  DISA  WESTHEM  to  determine  thrsteffCrequkemeS 
the  Defense  megacenters  The  implementation  of  DISA’s  internal  management  control 

FundraTVW  bf  ?1SCUlSed  c  the  rep°rt  on  **  audit  of  thQ  Defense  Business  Operations 
Defense  Inf(?rmatl0n  Services  Organization  Financial  Statements  for  FY  1994  and 

0n  °f  1116  Mana*eme“  °f  Megacato,  ft 

Potential  Benefits  of  Audit.  Recommendations,  when  implemented  will  nrnvid** 
measurable  workload  factors  to  use  in  determining  staffZP  SSuSenTs  accurate 
staffing  requirement  estimates  for  the  Defense  megacenters  and  accumte  oroiecS  nf 
costs  and  benefits  for  the  consolidation  of  data  processing  facilities  (see  Appendix  C). 

Summary  of  Recommendations.  We  recommend  that  in  determining  staffing 
requirements  for  the  Defense  megacenters,  the  Director,  DISA  Western  Hemispher? 

which  °the0rndm  Performed  bY  computer  personnel  rather  than  the  speed  at 

which  the  computer  processes  an  instruction;  revise  the  staffing  requirement  estimates 

budg?S  for  Ae  Defense  megacenters  based  on  using  measurable 
nrnW^ and  postPone  Panting  the  cost  analysis  to  Congress  until  the 
projected  staffing  requirements  are  revised  using  measurable  workload  factors. 


Management  Comments.  DISA  disagreed  with  our  assessment  of  the  method  it  used 
to  determine  estimates  of  staffing  requirements  for  the  Defense  megacenters.  DISA 
stated  that  it  determined  the  staffing  requirements  for  the  Defense  megacenters  based  on 
a  method  that  used  work  load  derived  from  how  fast  the  computer  processes 
instructions  because  that  method  is  the  only  reasonable  alternative  available.  DISA 
used  the  KMPG  Peat  Marwick  "Best  Data  Center  Practices"  Study  (the  Study)  to 
support  its  position  that  millions  of  instiuctions  per  second  is  an  appropriate  method  for 
determining  staffing  requirements.  A  discussion  of  management  comments  is  in  Part  II 
of  the  report,  and  the  complete  text  of  the  comments  is  in  Part  IV. 

Audit  Response.  We  disagree  with  DISA's  position  that  work  load  derived  from  how 
fast  the  computer  processes  instructions  is  the  only  reasonable  workload  factor  available 
for  determining  staffing  requirements  for  the  Defense  megacenters.  DISA  incorrectly 
applied  the  results  of  the  Study  to  support  its  use  of  that  workload  factor  in  determining 
staffing  requirements.  We  request  that  DISA  provide  additional  comments  on  the  final 
report  by  April  10,  1995. 
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Part  I  -  Introduction 


Introduction 


Background 


Defense  Information SInfrastructure  S  ^^Def”  ^  Manager  of 

Decision 918,  September  15  1992  designation^  i*6  Management  Report 
Agency  (DISA)P  as  le  ’cenSi  mTger'  Systems 

Infrastructure.  In  that  capacity  DISA  is  resnnndhi.  ?  •  ?efense  formation 
security,  standards  long-haul  Lffp  ble  for  mformation  technology 
certification,  and  data  processing  faci?h^UI1m!?AnS’  ,  telecommunications 
Information  Services  oStioTi  nTQ  A  es5blished  the  Defense 
(WESTHEM),  to  manage  S  J™  ■ DIS^  .,.Western  Hemisphere 

information  technology  se^ices  to  DoD^ustomel  nl10,  pr°vid® 

and  coordinated,  with  the  FY 19Q3  r™.™--  ln  iyyi;’  DISA  developed 
Realignment,  the  DoD  Data  Cenwrrfnc^T18^11  on  Base  Closure  and 
to  consolidate  data  processing  facilities0 /ntntlC>i a  rw  Consolidation  Plan) 
megacenters  will  pS The 
customers.  DISA P  WESTHEM  efforts  m  pr°CfSUlg  for  DoD 

16  Defense  megacenters  beg  Sin  the  S  quarter^FYlOQ1?6  int° 

estimated  for  the  fourth  quarter  of  FY  1997  ^  ^  1993  with  completion 

S"rect0“d^  ^  a  »*“ 

processing  facilities.  assistance  in  auditing  the  newly  consolidated  data 


Objectives 


The  announced  audit  objective  was  to  evaluate  app*  • 

of  operations  of  the  Defense  Megacenter  fDMn  £ICTienc.y  effectiveness 

objective  for  this  segment  nf  .  .  nier  (DMC)  St.  Louis,  Missouri.  The 

DISA  WKTH™ T  dete^e  toe  Si'°  na,mK  ,he  selected  by 

megacenters.  We  also  evXated  oW™f  g  ft,““‘ements  for  the  Defense 

WESTHEM  methodology  for  det^bSLiZStf  SfCabIe  ‘  •  the  DISA 
subsequent  report  will  discuss  the  effiriJL,,  ?taffing  requirements.  A 

of  DMC  St.  Louis, Missouri  V  and  effectlvenes«  of  the  operations 


Scope  and  Methodology 


WESTHEM^sed^n  determSng ' th?stafrablllty  °f  the  workIoad  factor  DISA 
DISA  WESTHEM  used  ™ame  woJwoad  for.DMC  St.  Louis, 

requirements  for  the  other  ,5  Defeme 


'  Formerly,  Defense  Information  Technology  Services  Organization. 

2 


Swt  MSA  WESTOPM1'3?'  T3  thr0Ugh  November  1994,  that 
_me  L,lbA  WESTHEM  development  of  staffing  reauirements 

r,,oa^  factor  expressed  in  millions  of  instructions  per  second  (MIPS)  and 

workload  factors  expressed  in  computer  output  function.  We  dW  ^  evaluate 

the  contractor-proprietary  model  that  computed  the  staffing  requirements 

Audit  Methodology.  Technical  staff  from  the  Inspector  General  DoD-  DISA- 
National  Defense  University;  Army;  Navy;  and  Air  Force  provided  assistance 

thpCna  f atmg  ^  workload  factor  usecl  in  determining  staffing  requirements  at 
?  ffSe  “^centers.  The  Quantitative  Methods  Division,  Audit  PlanSng 

Audithiff1111 T^!d  UP^01^  ■  P^ec^°^e  ’  0ffice  of  the  Assistant  Inspector  Generalfo? 
Auditmg  DoD,  provided  assistance  m  reviewing  the  KPMG  Peat  Marwick- 

M  pf  CCnter  ?a?icf !  Study-  Also>  Ac  Technical  Assessment  DiSn 
Audit  Planning  and  Technical  Support  Directorate,  Office  of  the  Assistant 

TSeftSf  lJ0rhAUditto8-  Do?-  obtatoed  Motion™  worUoad  SS 

nd  methods  used  by  commercial  computer  firms  in  determining  staffing 

te3pu“ers”4dabvPSenSg  faCUi,y'  ThreC  0f  the  four  f™s  -anufactur! 
uie  computers  used  by  the  Defense  megacenters,  and  one  firm  uses  similar 

computer  models.  The  Technical  Assessment  Division  also  provided  definitions 

and  explanations  of  technical  computer  terms.  Also,  we  obtained  similar 

wSno\on  worW°ad  factors  and  stafr“>*  methods  ^  by  to Amy  F™Ss 
Integration  Support  Agency,  the  Air  Force  Management  Engineering  Agencv 

Manpow®r  Information  Resource  Management  Division2  and  the 
DISA  Management  Analysis  and  Internal  Review  Division.  We  did  not  relv  on 

0bSTPr0CeSSed  ““  0r  sta,istical  sampling  procedures  <0  acSeve  te  LS 

Audit  Period,  Standards,  and  Potential  Benefits.  We  performed  the  audit 

t°Z  aZ  tS0Ugh  0ctober  1994  at  Ae  organizations  listed  in  Appendix  D  We 
performed  this  economy  and  efficiency  audit  in  accordance  with  anriitino 
standards  issued  by  the  Comptroller5^  General  oTSTi K  States  af 

by?C  peCtor  GeneraI>  DoD  We  Acluded  such  tests  ofintemal 
conaSd duTg  telSk  eSSMy'  Append“  D  liStS  ,he  o-sanizatiohs  visited  or 


Internal  Controls 


We  evaluated  the  internal  controls  applicable  to  the  DISA  WFSTHfa/t 
methodology  for  determining  estimates  of?taffin|  requirements  for  toe  dS 
megacenters.  We  identified  no  material  internal  control  weaknesses. 

We  will  discuss  DISA  WESTHEM's  implementation  of  its  internal  management 
Fund^ffrrfi311! thC  trep0rtc0n  the  audit  of  Ae  Defense  Business  Operations 
F^19^4^e^roiectOn4RE°20n^erV1Cew  0rg^Azafi°n  Financial  Statement  for 
implementationlof^ts^inrernal^rnanag^enuiontro'l^rogram^n  thef  reportondte 

"Note4&4m0r  °f  the  Megacen,lr-  S* 
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Introduction 


Prior  Audits  and  Other  Reviews 

The  Defense  megacenter  staffing  requirements  have  not  been  audited  in  the  last 
o  years. 
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Part  II 


Finding  and  Recommendations 
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Determining  Staffing  Requirements  at 
the  Defense  Megacenters 

The  Defense  Information  Systems  Agency,  Western  Hemisphere  did  not 
use  a  good  methodology  to  determine  staffing  requirements  for  the 
16  Defense  megacenters. 

The  workload  factor,  millions  of  instructions  per  second  (MIPS),  has  no 
direct  relationship  to  workload  functions,  such  as  computer  tape 
changes,  console  commands,  input  and  output  functions,  or  paper 
changes,  performed  by  computer  personnel.  MIPS  is  an  expression  of 
how  fast  the  computer  central  processing  unit  can  process  instructions. 
In  addition,  using  MIPS  to  compute  staffing  requirements  is  inconsistent 
with  the  methods  used  by  DoD  and  commercial  organizations  to  compute 
staffing  for  a  data  processing  facility. 

As  a  result,  the  projected  staffing  requirements,  which  are  not 
supportable  for  the  16  Defense  megacenters,  could  lead  to  either 
overstaffing  or  understaffing  of  the  Defense  megacenters.  In  addition, 
the  cost  analysis  showing  $183  million  in  personnel  costs  and  benefits 
from  the  consolidations  into  Defense  megacenters  needs  revision  prior  to 
presentation  to  Congress  in  March  1995. 


Background 


Definition  of  MIPS,  MIPS  represents  the  number  of  instructions  the  central 
processing  unit  (CPU)  of  a  computer  can  process  in  a  second.  MIPS  defines  the 
speed  of  a  CPU  based  on  its  architecture  and  operating  speed.  A  MIPS  rating 
for  a  CPU  represents  the  throughput2  of  the  processor  and  is  determined  by 
running  a  computer  analysis  software  program.  The  software  program  takes 
into  account  the: 

o  peripheral  devices  connected  to  the  CPU, 
o  number  of  devices, 

o  internal  communications  between  CPU  and  devices, 
o  external  communications  connecting  the  user  to  the  CPU,  and 
o  type  and  configurations  of  operating  systems  software. 


2  Throughput  is  the  rate  at  which  a  request  (instruction)  is  answered  by  the 
CPU.  CPU  throughput  is  measured  in  MIPS. 
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MIPS  does  not  measure  the  work  load  performed  by  computer  personnel  at  a 
data  processing  center.  Rather,  MIPS  expresses  the  speed  at  which  a  computer 
processes  instructions. 

Variances  in  MIPS  Ratings  for  the  Same  Model  of  Computer.  A  MIPS 
rating  for  a  specific  model  of  computer  will  vary  among  data  processing 
facilities  that  use  the  same  model  of  computer.  A  MIPS  rating  will  vary 
because  of  various  computer  configurations  and  software  analysis  programs  that 
determine  MIPS  ratings. 

The  software  analysis  programs  used  by  the  computer  manufacturer  and  third 
party  organizations  could  provide  different  MIPS  ratings  for  the  same  CPU 
because  of  differing  computer  configurations.  Also,  the  software  analysis 
programs  frequently  do  not  consider  the  complexity  and  efficiency  of  the 
functional  applications  operating  on  the  CPU. 

Work  Load  Performed  by  Personnel  at  a  Data  Processing  Facility. 

Measurable  work  load  performed  by  personnel  at  a  data  processing  facility  is 
composed  of  those  tasks  or  functions  that  produce  a  consistent  and  verifiable 
output.  The  types  of  output  include  computer  tape  changes,  console  command, 
input  and  output,  or  paper  changes.  Staffing  requirements  vary  with  the  number 
and  type  of  tasks  or  functions  performed  within  a  given  time  period.  For 
example,  personnel  at  data  processing  facilities  perform  one  to  three  types  of 
computer  tape  changes.  The  types  of  computer  tape  changes  determine  the  type 
and  number  of  personnel  needed  to  perform  these  tasks. 

o  Personnel  are  needed  to  physically  locate,  mount,  thread,  and  start 
each  round  tape  change. 

o  Personnel  are  needed  to  physically  locate  and  insert  tape  cassettes  into 
the  tape  reader. 

o  Personnel  are  not  needed  to  operate  robotic  tape  drives;  however,  the 
console  operator  monitors  robotic  tape  drives. 


DISA  WESTHEM  Basis  in  Determining  Staffing 
Requirements 


Use  of  MIPS  as  the  Workload  Factor  in  Determining  Staffing 
Requirements.  DISA  WESTHEM  used  MIPS  because  DISA  WESTHEM 
viewed  MIPS  as  the  only  single  common  denominator  among  all  16  Defense 
megacenters.  Additionally,  because  it  determined  that  other  alternatives  were 
not  workable,  DISA  WESTHEM  decided  to  use  MIPS. 

For  example,  DISA  WESTHEM  did  not  use  the  services  of  the  DISA 
Management  Analysis  and  Internal  Review  Division  (Management  Analysis 
Division),  which  performs  studies  of  staffing  requirements  for  DISA.  In 
January  1993,  DISA  WESTHEM  discussed  the  determination  of  staffing 
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requirements  at  the  Defense  megacenters  with  personnel  from  the  Management 
Analysis  Division.  DISA  WESTHEM  believed  that  historical  data  needed  by 
the  Management  Analysis  Division  were  not  available  and  would  take  too  long 
to  obtain.  However,  we  confirmed  with  representatives  from  the  Management 
Analysis  Division  and  DMC  St.  Louis  that  historical  data  were  available  and 
that  staffing  requirements  could  have  been  determined  within  1  year.  DISA 
WESTHEM  issued  the  preliminary  staffing  requirements  in  January  1994, 
revised  them  in  September  1994,  and  finalized  them  in  January  1995. 
Appendix  A  provides  details  on  the  development  of  the  staffing  requirements  at 
the  Defense  megacenters. 

Concerns  on  Using  MIPS  in  Determining  Staffing  Needs.  Representatives 
from  DISA  WESTHEM  Computer  Operations  Support  Directorate,  Capacity 
Management  Division;  Defense  Megacenters  St.  Louis  and  Huntsville;  and 
Resource  Management  Division  were  concerned  about  using  MIPS  as  the 
workload  factor  in  determining  staffing  requirements  at  the  Defense 
megacenters.  Those  concerns  included  the  following. 

o  MIPS  is  not  an  acceptable  workload  factor  because  MIPS  is  a 
measurement  that  widely  fluctuates,  depending  on  the  work  load  processed  and 
the  individual  or  organization  that  provides  the  MIPS  rating.  Appendix  B 
illustrates  the  MIPS  ratings  used  by  DISA  WESTHEM,  DMC  St.  Louis,  and  a 
commercial  organization  for  the  same  model  of  computer. 

o  Work  load  expressed  in  MIPS  has  no  measurable  elements. 

o  Numerous  factors,  such  as  type  of  processing,  customer  services 
provided,  and  variance  in  processing  environments  should  be  considered  in 
determining  staffing  requirements. 

o  MIPS  does  not  correlate  to  measurable  workload  outputs  produced  by 
a  data  processing  facility. 

o  MIPS  does  not  account  for  the  operations  of  the  data  processing 
facility.  Some  of  the  operational  elements  included  tape  mount  changes,  print 
changes,  input  and  output  functions,  amount  of  automation,  and  the  kinds  of 
applications  and  data  bases  being  used. 

o  DISA  WESTHEM  did  not  document  the  decision  justifying  the  use  of 
MIPS  for  staffing  requirements. 

Relationship  of  the  MIPS  Ratings  to  Staffing  Requirements.  DISA 
WESTHEM  estimated  that  for  every  two  MIPS  processed,  one  person  was 
needed.  However,  advancements  in  information  technology  that  provide  for 
more  processing  capability  do  not  necessarily  increase  staffing  requirements. 
Also,  robotic  tape  functions  would  require  fewer  staff  than  manual  tape 
functions.  Conversely,  less  processing  capability  does  not  necessarily  reduce 
staffing  requirements.  The  complexity  of  the  functional  applications  processed 
on  the  computer  and  the  associated  workload  functions  could  result  in  an 
increase  in  personnel. 
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Staffing  requirements  vary  with  the  number  and  type  of  tasks  or  functions 
performed  at  a  data  processing  facility  within  a  given  time  frame.  By  using  a 
work  load  factor  that  does  not  measure  functions  performed  by  personnel  at  a 
data  processing  facility,  the  megacenters  could  be  either  overstaffed  or 
understaffed.  Understaffed  facilities  cannot  perform  their  missions  and 
functions  effectively,  and  overstaffed  facilities  cannot  provide  efficient 
operations. 


Military  Departments'  Methods  for  Determining  Staffing 
Requirements 


Army's  Method  In  Determining  Staffing  Requirements  at  a  Data  Processing 
Facility.  The  mission  of  the  Army  Forces  Integration  Support  Agency  (Support 
Agency),  Special  Action  Division,  is  to  perform  surveys  or  reviews  of  staffing 
requirements  for  organizations  that  are  unique  in  design  or  mission.  The 
Support  Agency's  method  for  determining  work  load  and  staffing  requirements 
is  based  on  measurable  workload  functions  and  the  number  of  individuals 
needed  to  support  those  functions.  A  representative  of  the  Support  Agency  gave 
the  following  examples  of  the  quantitative  historical  data  on  some  data 
processing  functions  that  would  be  needed  to  determine  the  number  of 
individuals  needed  to  support  a  data  processing  facility: 

o  the  number  of  computer  tape  mounts  and  how  long  it  takes  to  mount  a 

tape; 


o  the  number  of  paper  changes  and  how  long  it  takes  to  change  the 

paper; 


o  the  number  of  system  failures,  type  of  failures,  number  of  failures, 
and  the  amount  of  time  taken  to  make  the  system  operational;  and 

o  the  amount  of  time  the  system  is  down  and  no  work  is  processed. 

Using  MIPS  as  a  workload  factor  is  inconsistent  with  the  Army's  method  for 
determining  staffing  requirements  at  a  data  processing  facility. 

Navy's  Method  for  Determining  Staffing  Requirements.  The  Navy  Total 
Force  Manpower  Requirements  Handbook  (the  Handbook)  outlines  the  basic 
methods  and  techniques  for  determining  staffing  requirements  for  the  Navy 
Total  Force  Manpower  Requirements  Program.  The  Handbook  does  not 
specifically  discuss  staffing  requirements  for  a  data  processing  facility. 
However,  the  Handbook  provides  guidelines  for  the  selection  of  potential 
workload  factors  and  defines  a  workload  factor  as  a  unit  of  measure  that  is 
consistently  relatable  and  predictable  to  the  work  needed  to  accomplish 
organizational  responsibilities. 
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Specifically,  workload  factors  should  be: 

o  directly  related  to  the  time  and  effort  expended  on  the  task; 

o  susceptible  to  audit,  so  that  the  accuracy  of  the  work  load  can  be 
readily  verified; 

o  clearly  identifiable  when  production  is  in  process  or  has  been 
completed;  and 

o  individually  standardized  in  terms  of  the  procedures  required  to 
complete  production. 

The  definition  of  MIPS  does  not  meet  the  Navy's  criteria  for  a  workload  factor 
used  in  determining  staffing  requirements. 

Air  Force's  Method  for  Determining  Staffing  Requirements.  Air  Force 
Regulation  25-5,  "Air  Force  Management  Engineering  Program  Policies, 
Responsibilities,  and  Requirements,"  May  16,  1988,  provides  details  on  the  Air 
Force's  Functional  Review  Process.  Even  though  Air  Force  Regulation  25-5 
does  not  specifically  discuss  staffing  of  a  data  processing  facility,  the  Regulation 
defines  the  major  characteristics  of  a  potential  workload  factor.  The  major 
characteristics  of  a  potential  workload  factor  follow. 

o  Collectibility-how  easily  a  unit  can  be  identified,  counted,  reported, 
and  validated. 

o  Relatability-how  well  the  workload  factor  logically  relates  to  the 
resources  required  to  complete  the  work. 

o  Programmability— how  well  the  workload  factor  predicts  required 
resources  for  future  time  periods. 

The  characteristics  of  MIPS  do  not  meet  the  Air  Force's  criteria  for  workload 
factors  used  in  determining  staffing  requirements. 


Commercial  Methods  for  Determining  Staffing  at  Data 
Processing  Facilities 


Methods  Commercial  Information  Processing  Organizations  Use  in 
Determining  Staffing  Requirements.  Commercial  information  processing 
organizations  generally  do  not  use  MIPS  to  determine  the  staffing  requirements 
at  a  data  processing  facility.  We  contacted  the  following  commercial 
information  processing  organizations  because  they  manufacture  or  use  the  same 
model  computers  that  are  in  operation  at  the  Defense  megacenters: 

o  Systems  Information  Support  Center,  International  Business 
Machines,  Incorporated  (IBM),  Tucson,  Arizona; 
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o  IBM  Government  Business  Office,  Washington,  D.C.; 

o  Loral  Federal  Systems,  Boulder,  Colorado;  and 

o  Amdahl  Federal  Service  Corp.,  Reston,  Virginia 

The  consensus  among  those  organizations  on  the  approach  in  determining 
starring  requirements  is  to  analyze  all  the  major  operations  and  requirements  of 
the  computer  center.  Such  an  approach  requires  an  understanding  of  the 
operations  and  products  of  a  mainframe  environment,  which  is  essential  for 
estimating  the  staffing  requirements  that  meet  customer  needs.  The  approach 
includes  the  review  and  analysis  of: 

o  operator  functions  (tape  mounts,  configuration  changes,  print 
management);  6  ’  F 


o  installation  data  storage  capacity; 
o  number,  type,  and  complexity  of  software  applications; 
o  operating  system  and  application  software  stability  and  reliability; 
o  user  interface  requirements;  and 

u  J  o  technology  enhancements  (automatic  tape  handling  systems,  modem 
hands-off  production  capabilities,  or  automated  lights-out4  operations). 

Using  MIPS  in  determining  staffing  requirements  is  inconsistent  with 
commercial  practices,  except  as  discussed  below. 


Using  MIPS  in  Determining  Staffing  Requirements.  The  Amdahl 
representative  stated  that  the  use  of  MIPS  as  the  primaiy  factor  of  staffing 
requirements  is  appropriate  only  when  usage  is  based  on  historical  data  and  the 
basic  operations  have  not  changed.  However,  operations  at  the  Defense 
megacenters  are  in  a  constant  state  of  flux  as  a  result  of  ongoing  consolidation 
efforts.  DISA  WESTHEM  projected  workload  data  using  MIPS;  the  data  were 
based  on  available  production  data  at  16  Defense  megacenters 


Projected  Staffing  Requirements  to  be  Presented  to  Congress 

The  House  of  Representatives  Conference  Report  103-624,  July  27,  1994 
directs  DISA  to  submit  to  the  House  and  Senate  Committees  on  Appropriations 
by  March  1,  1995,  a  cost  analysis  and  review  results  of  the  potential  costs  and 
benefits  associated  with  the  consolidation  for  each  Defense  megacenter.  DISA 
is  required  to  include  in  the  cost  analysis  and  review  results  an  explanation  of 


^  Hands-off  means  functions  that  do  not  require  operator  intervention. 

4  Lights  out  operations  do  not  require  personnel  24  hours  a  day  7  days  a  week. 
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the  difference  between  the  July  1993  estimate  of  $309  million  for  DoD's  5-year, 
one-time  consolidation  cost  and  the  $417  million  cost  estimate  in  the  FY  1995 
budget  justifications.  The  cost  analysis  and  review  results  will  include 
anticipated  personnel  costs  and  benefits  based  on  projected  staffing  requirements 
derived  from  using  MIPS  as  the  workload  factor.  If  the  projected  staffing 
requirements  are  not  revised  using  a  measurable  workload  factor,  the  anticipated 
personnel  costs  and  benefits  from  the  data  processing  facility  consolidations  will 
be  inaccurate. 


Conclusion 


MIPS,  the  workload  factor  DISA  WESTHEM  used  in  determining  staffing 
requirements  at  a  data  processing  facility  is  not  a  measurable  workload  factor. 
MIPS  does  not  consider  operations  and  input  and  output  functions  of  computer 
personnel,  which  are  essential  in  determining  associated  staffing  needs.  The 
MIPS  methodology  for  determining  staffing  requirements  is  inconsistent  with 
the  estimating  techniques  used  by  the  Army,  Navy,  Air  Force,  and  commercial 
entities,  such  as  IBM  and  Loral.  As  a  result,  DISA  could  either  overstaff  a 
megacenter  and  create  inefficiencies  or  understaff  a  megacenter  and  cause  it  to 
operate  ineffectively.  Also,  in  the  near  term,  any  cost  analysis  based  on 
personnel  costs  and  benefits  that  were  computed  using  the  staffing  requirement 
estimates  derived  from  the  MIPS  calculation  will  be  inaccurate. 


Management  Comments  on  the  Finding  and  Audit  Response 


Management  Comments.  DISA  nonconcurred  with  the  finding.  DISA  stated 
the  audit  conclusion  that  the  use  of  MIPS  was  an  inappropriate  basis  for 
estimating  staffing  requirements  at  the  Defense  megacenters  is  contrary  to  the 
conclusion  reached  by  KPMG  Peat  Marwick  (Peat  Marwick)  in  its  "Best  Data 
Center  Practices"  study  (the  Study).  DISA  also  stated  that  although  using  MIPS 
to  determine  staffing  does  not  explicitly  recognize  technical  concerns  specified 
in  the  audit  report,  using  MIPS  in  determining  staffing  requirements  does 
recognize  that  the  Defense  megacenters  must  become  as  efficient  as  the 
commercial  competition  to  remain  in  business. 

In  1992,  Peat  Marwick  prepared  the  Study  for  DISA  to  document  how  the  best 
performing  data  processing  facilities  in  the  commercial  world  operate.  DISA 
stated  that: 

o  the  research  performed  by  Peat  Marwick  conclusively  showed  that 
MIPS  is  an  appropriate  workload  measurement  for  estimating  staffing 
requirements  for  mainframes, 

o  Peat  Marwick  calculated  work  load  as  a  function  of  MIPS  used  by  the 
data  processing  facility,  and 
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o  the  research  performed  by  Peat  Marwick  established  the  correlation 
between  MIPS  used  by  a  data  processing  facility  and  other  data  processing 
facility  workload  functions. 

DISA  stated  that  in  the  absence  of  other  measures  related  to  functions  performed 
by  personnel,  MIPS  can  be  used  to  estimate  the  total  staffing  required  to  process 
a  specified  work  load. 

DISA  stated  that  internal  organizational  turmoil  had  inhibited  determining 
staffing  requirements  based  on  measurable  workload  factors  and  that  a  review 
using  tools  that  measure  workload  volume,  accomplishment  times  and 
personnel  unit  cost  per  output  could  now  be  accomplished.  DISA  has  scheduled 
reviews  for  FY  1995  to  attempt  to  develop  interim  staffing  requirements  based 
on  measurable  workload  factors.  If  the  effort  is  successful,  the  methodology 
using  measurable  workload  factors  may  be  applied  to  other  Defense  megacenters 
that  are  stable  but  not  yet  optimized."  The  complete  text  of  management 
comments  is  in  Part  IV. 

Audit  Response.  We  disagree  that  the  Peat  Marwick  Study  concluded  that 
MIPS  could  be  used  as  a  workload  factor  in  determining  staffing  requirements 
at  a  data  processing  facility.  In  fact,  the  Study  did  not  intend  to  provide 
statistically  proven  relationships  between  the  number  of  MIPS  used  by  a  data 
processing  facility  and  staffing  required  for  a  data  processing  facility.  Rather, 
in  the  Study,  Peat  Marwick  provided  a  quantified,  but  notional  description  of 
best  practices  observed  at  more  than  150  computer  operations  in  the  commercial 
environment.  Peat  Marwick  derived  those  best  practices  using  data  maintained 
in  its  Automated  Peer  Group  Comparison  data  base.  The  Study  stated: 

"•  ■  We  believe  that  Automated  Peer  Group  Comparison  clients 
should  only  use  benchmark  data  to  identify  opportunities  for 
improvements  at  the  data  center  being  evaluated  ....  Therefore,  it  is 
important  to  not  read  too  much  into  the  numbers  themselves  but  to 
recognize  the  value  they  play  as  performance  indicators  .  .  .  ." 

[emphasis  added] 

While  the  typical  DoD  data  center  may  be  inherently  different  from 
commercial  data  centers  in  some  respects,  it  is  hoped  that  many  of  the 
suggested  "best  practices"  discussed  in  this  report  can  be  readily 
adopted  by  DoD  .  .  .  KPMG  Peat  Marwick  developed  the  Automated 
Peer  Group  Comparison  tool  in  order  to  effectively  perform 
benchmarking  and  develop  performance  indicators  (or  metrics)  for  our 
clients'  data  centers  .  .  .  Benchmarking  allows  the  evaluator  to 
identify  opportunities  for  potential  improvements  .  .  .  . " 

DISA's  assertion  that  the  Study  provided  empirical  metrics  correlating  staffing 
to  work  load  based  on  MIPS  is  incorrect.  Peat  Marwick  provided  data  on 
staffing  ratios  based  on  MIPS  used  by  a  data  processing  facility  to  demonstrate 
that  large  data  processing  facilities  are  more  efficient  than  small  data  processing 
facilities.  DISA  inappropriately  used  the  Study  to  correlate  staffing 
requirements  for  a  data  processing  facility  defined  as  "average"  to  staffing 
requirements  for  a  data  processing  facility  defined  as  "best  practice."  DISA 
incorrectly  assumed  that  a  "best  practice"  data  processing  facility  had  a  lower 
staffing  need  than  the  "average"  site  because  of  the  MIPS  used  by  the  data 
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processing  facility  rather  than  because  of  other  factors,  such  as  hardware  and 
software  configurations.  DISA's  assertion  that  the  Study  conclusively  related 
staffing  to  work  load  based  on  MIPS  is  also  flawed  because: 

o  the  significant  difference  between  the  staffing  needs  at  the  "average" 
site  and  the  "best  practice"  site  indicates  that  the  underlying  data  are  highly 
variable  with  no  statistically  significant  or  proven  correlation; 

o  not  one  of  the  companies  whose  data  was  used  to  compile  the  data  in 
the  "Staffing  Based  on  Workload"  chart  was  a  totally  "average"  site  or  a  totally 
"best  practice"  site,  which  is  needed  for  a  statistically  significant  and  proven 
relationship;  and 

o  the  Study  does  not  explain  the  specific  practices  needed  in  each 
functional  area  for  a  data  processing  facility  to  achieve  the  staffing  ratios  shown 
in  the  chart,  which  are  necessary  for  a  statistically  significant  and  proven 
relationship. 

Again,  the  premise  that  MIPS  is  an  appropriate  workload  factor  for  determining 
staffing  requirements  at  the  Defense  megacenters  is  flawed.  As  stated  in 
comments5  from  the  National  Defense  University,  Department  of  Defense,  on 
the  Peat  Marwick  Study  and  on  our  draft  audit  report: 

"...  MIPS  is  a  measure  of  CPU  (Central  Processor  Unit)  capability. 

It  is  one  of  many  such  measures  which  have  proven  invalid  for 
comparing  machines  of  different  architectures,  selecting  a  computer 
for  a  particular  suite  of  applications  or  determining  the  structure  of 
the  staff  needed  to  execute  that  suite  of  applications.  Any  detailed 
study  of  DPI  operations  will  demonstrate  sensitivity  to  specific 
combinations  of  equipment,  staffing  mix,  and  the  application  portfolio 
.  .  .  Simple  measures  of  workload  such  as  used  MIPS  cannot  capture 
this  interaction.  As  a  result,  MIPS  has  a  bad  reputation  among 
manpower  organizations,  acquisition  organizations  and  equipment 
manufacturers  .  .  .  Detailed  configuration  studies  are  needed  to  select 
equipment  and  prepare  job  descriptions  .  .  .  ." 

Accordingly,  DISA  did  not  use  an  appropriate  methodology  to  determine  the 
staffing  requirements  at  the  Defense  megacenters. 


Recommendations,  Management  Comments,  and  Audit 
Response 

Changes  to  Recommendations.  We  revised  Recommendations  2.  and  3.  to 
clarify  the  actions  to  be  taken  after  DISA  revises  the  staffing  requirements  for 
Defense  megacenters.  Therefore,  we  request  that  the  Director,  Defense 


5  DISA  requested  that  the  National  Defense  University  comment  on  the  Study 
and  on  our  draft  audit  report. 
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Information  Systems  Agency  provide  comments  on  the  revised 
recommendations  in  response  to  the  final  report. 

We  recommend  that  the  Director,  Defense  Information  Systems  Agency, 
Western  Hemisphere: 

1.  Use  measurable  workload  functions  that  are  performed  by  computer 
personnel  rather  than  the  speed  in  which  the  computer  processes  an 
instruction  (as  expressed  in  millions  of  instructions  per  second)  in 
determining  the  staffing  requirements  for  the  16  Defense  megacenters. 

Management  Comments.  DISA  nonconcurred  with  the  recommendation, 
stating  that  the  only  reasonably  accurate  measurement  it  has  on  future  operations 
at  the  Defense  megacenters  is  total  projected  work  load  measured  in  MIPS. 
DISA  also  stated  that  the  methodologies  described  in  our  report  that  could  be 
used  to  estimate  the  staffing  requirements  require  quantitative  historical  data  to 
produce  staffing  estimates  and  that  the  earliest  the  required  information  would 
be  available  is  the  end  of  FY  1995.  DISA  intends  to  continue  developing  final 
staffing  requirements  using  projected  work  load  in  MIPS.  DISA  also  stated  that 
it  will  attempt  to  develop  interim  staffing  requirements  based  on  measurable 
workload  factors  to  include  workload  volume,  accomplishment  times,  and 
personnel  unit  cost  per  output. 

Audit  Response.  As  discussed  in  the  prior  audit  response,  work  load  measured 
in  MIPS  is  not  an  appropriate  basis  for  determining  staffing  requirements  for  the 
Defense  megacenters.  Additionally,  in  January  1993,  DISA  WESTHEM 
contacted  the  DISA  Management  Analysis  Division  about  developing  staffing 
requirements  for  the  Defense  megacenters,  but  chose  not  to  have  the 
Management  Analysis  Division  determine  the  staffing  requirements  for  the 
Defense  megacenters.  It  is  true  that  building  regression  models  to  determine 
appropriate  relationships  requires  historical  data;  however,  none  exist  in  exactly 
the  size  and  form  needed  for  the  16  new  Defense  megacenters.  Therefore,  a 
prospective  approach  using  engineered  work  measurement  standards  for  staffing 
could  be  used.  But  more  important,  DISA's  use  of  MIPS  to  estimate  staffing 
requirements  puts  DoD  at  risk  of  having  Defense  megacenters  that  are  not 
competitive  because  they  are  either  understaffed  and  not  effective  or  overstaffed 
and  not  efficient. 

2.  Revise  the  staffing  requirements  for  the  16  Defense  megacenters  based 
on  using  the  measurable  workload  factors  discussed  in  Recommendation  1 
and  make  associated  budget  adjustments  in  the  spring  1996  Program 
Objective  Memorandum. 

Management  Comments.  DISA  nonconcurred  with  the  draft  report 
recommendation.  DISA  will  continue  to  develop  the  final  staffing  requirements 
using  projected  work  load  in  MIPS  because  no  other  viable  option  is  available. 
DISA  will  attempt  to  use  measurable  workload  factors  in  FY  1995  to  establish 
interim  staffing  requirements  during  the  period  in  which  work  load  transitions  to 
the  megacenters. 
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Audit  Response.  If  DISA  is  able  to  use  measurable  workload  factors  to 
determine  the  interim  staffing  requirements,  DISA  should  use  a  similar  method 
for  estimating  the  final  staffing  requirements.  Therefore,  in  determining 
staffing  requirements,  DISA  should  use  a  methodology  that  uses  measurable 
workload  functions.  We  revised  the  recommendation  so  that  DISA  WESTHEM 
will  make  the  concomitant  budget  adjustments  derived  from  estimating  staffing 
requirements  based  on  measurable  workload  factors. 

3.  Postpone  presenting  to  Congress  the  cost  analysis  and  review  results  of 
the  potential  costs  and  benefits  associated  with  the  consolidation  of  the 
16  Defense  megacenters  until  the  projected  staffing  requirement  estimates 
have  been  revised  as  prescribed  in  Recommendations  1  and  2. 

Management  Comments.  DISA  nonconcurred  with  the  draft  report 
recommendation.  DISA  stated  that  it  will  be  using  the  most  accurate  staffing 
requirements  available  at  the  time  the  cost  analysis  is  presented  to  Congress  in 
March  1995. 

Audit  Response.  If  DISA  uses  staffing  requirements  that  are  based  on  MIPS 
instead  of  measurable  workload  factors,  the  cost  analysis  scheduled  for 
presentation  to  Congress  in  March  1995  will  not  be  accurate.  We  revised  the 
recommendation  because  we  believe  DISA  should  delay  providing  the  cost 
analysis  until  a  methodology  based  on  measurable  workload  standards  is  used  to 
estimate  staffing. 
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Appendix  A.  DISA  WESTHEM  Development  of 

Personnel  Staffing  Requirements 
for  the  Defense  Megacenters 


The  Defense  Information  Systems  (DISA)  Western  Hemisphere  (WESTHEM) 
Realignment  and  Closure/Megacenter  Consolidation  Office  (Consolidation 
Office)  is  responsible  for  determining  the  interim  and  final  staffing  requirements 
for  each  Defense  megacenter.  The  interim  staffing  requirement  is  the  number 
of  personnel  needed  after  the  Military  Departments  complete  the  transfer  of 
work  load  to  a  Defense  megacenter  and  before  automation  commences.  The 
final  staffing  requirement  is  the  number  of  personnel  needed  after  automation 
commences.  DISA  WESTHEM  estimated  the  millions  of  instructions  per 
second  (MIPS)  processed  at  a  Defense  megacenter  and  used  MIPS  as  the  basis 

StjSSS??11?  assoJciat?,d  st?ffing  requirements.  To  estimate  the  MIPS,  DISA 
STHEM  obtained  utilization  data,  available  production  data,  average  central 
processing  unit  (CPU),1  average  peak  CPU,  and  high  peak  CPU*  for  all 
16  Defense  megacenters.  The  Computer  Performance  Management  Division 
Pensacola,  Florida,  collected  the  utilization  data  and  converted  it  to  MIPS  to 
JrTvJr'0P  the  Preliminary  Sizing  Studies  (the  Studies).  The  Studies  estimated 
MiFb  tor  International  Business  Machines,  Incorporated  (IBM),  IBM 
compatible,  and  Unisys  mainframe  systems  at  the  Defense  megacenters  and  for 
some  sites  designated  for  closure. 

I?e  DISAWESTHEM  Consolidation  Office  used  the  Electronic  Data  Systems' 
Plano  Staffing  Model  (the  Staffing  Model),  which  is  driven  by  a  workload 
factor  expressed  m  MIPS,  to  determine  the  staffing  requirements  for  the 
Defense  megacenters.  DISA  WESTHEM  entered  the  estimated  MIPS  obtained 
lroJJ  llie  Studies  into  the  Staffing  Model,  which  then  produced  the  estimated 
staffing  requirements  for  the  Defense  megacenter.  The  Staffing  Model  used  a 
rating  of  2.2  MIPS  per  person  to  calculate  the  staffing  requirements  at  the 
Defense  megacenters.  However,  DISA  WESTHEM  did  not  know  the 
assumptions  and  methodology  employed  by  the  model  in  calculating  the  staffing 
requirements  because  the  software  for  the  staffing  Model  is 
contractor-proprietary.  6 


1  Average  CPU  is  the  utilization  by  hour  for  each  month  of  data  collection 
,  Average  peak  CPU  is  the  average  CPU  utilization  of  the  busiest  20  percent  of 
the  usage  for  each  hour. 

3  Peak  CPU  is  the  CPU  utilization  of  the  1  hour  during  which  utilization 
was  highest. 
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Appendix  B.  Variances  in  MIPS  Ratings  Affect 

Work  Load  Expressed  in  MIPS 


Work  load  expressed  in  MIPS  varies  significantly  because  MIPS  ratings  are  not 
consistent.  Because  MIPS  ratings  widely  fluctuate,  MIPS  is  not  a  reliable 
workload  factor  for  determining  staffing  requirements  for  even  identical 
computer  models.  The  table  below  shows  the  variances  in  MIPS  ratings  and 
computer  work  load  expressed  in  MIPS. 

The  table  shows  three  MIPS  ratings  for  the  CPU  operating  at  DMC  St.  Louis  as 
of  September  1994.  We  obtained  the  three  MIPS  ratings  from: 

o  the  DMC  St.  Louis,  which  represents  the  rating  applied  by  DMC 
St.  Louis  employees  for  each  CPU; 

o  the  WESTHEM  Study,  which  represents  the  MIPS  ratings  used  to 
develop  the  projected  work  load  for  the  DMC  St.  Louis  IBM  Sizing  Study 
Preliminary  Report;  and 

o  industry,  which  represents  ratings  determined  by  commercial 
computer  organizations  for  each  CPU  at  DMC  St.  Louis. 

The  utilization  percentages  in  the  table  were  shown  in  the  WESTHEM  Study. 
We  applied  those  percentages  to  the  MIPS  ratings  to  determine  the  work  load 
expressed  as  MIPS. 
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Variances  in  MBPS  Ratings 


Appendix  B.  Variances  in  MIPS  Ratings  Affect  Work  Load  Expressed  in  MIPS 
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Appendix  C.  Summary  of  Potential  Benefits 

Resulting  From  Audit 


Recommendation 

Reference  Description  of  Benefit  Type  of  Benefit 


1.  Economy  and  Efficiency.  Provides  Undeterminable.* 

staffing  requirements  for  the 

Defense  megacenters  based  on 
measurable  workload  factors. 

2.  Economy  and  Efficiency.  Requires  Undeterminable.* 

revising  staffing  requirements  for 

the  Defense  megacenters  and 
associated  budget  documents  to 
ensure  accuracy. 

3.  Economy  and  Efficiency.  Provides  Undeterminable.* 

Congress  accurate  and  supportable 

staffing  requirements  based  on 
implementation  of  Recommendation  1 . 

*DISA  will  be  able  to  determine  potential  monetary  benefits  upon  implementing 
the  recommendations  and  determining  staffing  requirements. 
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Appendix  D.  Organizations  Visited  Or  Contacted 


Department  of  the  Army 

U.S.  Army  Forces  Integration  Support  Agency,  Fort  Belvoir,  VA 

Department  of  the  Air  Force 

Air  Force  Management  Engineering  Agency,  San  Antonio,  TX 


Department  of  the  Navy 

Manpower  Information  Resource  Management  Division,  Chesapeake,  VA 


Defense  Organizations 

Defense  Information  Systems  Agency,  Washington,  DC 
Western  Hemisphere,  Denver,  CO 

Defense  Megacenter,  St.  Louis,  St.  Louis,  MO 
Defense  Megacenter,  Huntsville,  Huntsville,  AL 
Army  Information  Services  Center,  Fort  Ritchie,  MD 
Marine  Corps  Information  Services  Center,  Quantico,  VA 
Management  Analysis  and  Internal  Review  Division,  Arlington,  VA 
National  Defense  University,  Fort  McNair,  Washington,  DC 


Non-Government  Organizations 

Amdahl  Federal  Service  Corp. ,  Reston,  VA 
Electronic  Data  Systems,  Herndon,  VA 

Government  Business  Office,  International  Business  Machines,  Incorporated 
Washington  DC 

Loral  Federal  Systems,  Boulder,  CO 

Systems  Information  Support  Center,  International  Business  Machines,  Incorporated, 
Tucson,  AZ 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Comptroller) 

Under  Secretary  of  Defense  (Acquisition  and  Technology) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 

Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 

Deputy  Under  Secretary  of  Defense  (Comptroller/Management) 

Deputy  Under  Secretary  of  Defense  (Comptroller/Program/Budget) 


Department  of  the  Army 

Secretary  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Secretary  of  the  Navy 

Auditor  General,  Department  of  the  Navy 

Comptroller  of  the  Navy 


Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Defense  Organization 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Information  Systems  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 
Inspector  General,  Central  Imagery  Office 
Inspector  General,  National  Security  Agency 
Director,  Defense  Logistics  Studies  Information  Exchange 
President,  National  Defense  University 
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Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

National  Security  Division  Special  Projects  Branch,  Office  of  Management  and  Budget 
Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 


Chairman  and  Ranking  Minority  Member  of  Each  of  the  Following  Congressional 
Committees  and  Subcommittees 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
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Part  IV  -  Management  Comments 


Defense  Information  Systems  Agency  Comments 


DEFENSE  INFORMATION  SYSTEMS  AGENCY 

701  S.  COURT  HOUSE  ROAD 
ARLINGTON.  VIRGINIA  22204-2109 


t*rt*To.  inspector  General  rcu  |J9? 


MEMORANDUM  FOR  INSPECTOR  GENERAL,  DEPARTMENT  OF  DEFENSE 

ATTN:  Director,  Readiness  and  Operational  Support 

SUBJECT:  DoDIG  Draft  Audit  Report  on  Staffing  Requirements 

at  the  Defense  Megacenters 
(Project  No . 4RE-5034 . 00) 

Reference:  DoDIG  Report,  subject  as  above,  12  Dec  94 


1.  We  have  reviewed  the  subject  draft  audit  report  per  your 
request,  and  appreciate  your  assistance  in  reviewing  the 
operations  of  our  Megacenters.  By  using  millions  of  instructions 
per  second  (MIPS)  to  determine  staffing  requirements,  we  were 
attempting  to  measure  the  size  of  the  workload  and  not  the 
capacity  of  the  data  center.  The  organizational  turmoil  that 
this  Agency  has  undergone  over  the  last  two  years  has  inhibited 
the  systematic  conduct  of  Efficiency  Reviews  (ER) ,  and  the 
determination  of  staffing  requirements  based  on  measurable 
workload  factors.  DISA  is  now  at  a  point  where  the  routine, 
systematic  review  of  our  total  organization  can  be  accomplished, 
using  tools  which  measure  workload  volume,  accomplishment  times, 
and  personnel  unit  cost  per  output.  These  reviews  are  scheduled 
to  begin  in  FY  1995  with  a  prototype  effort  at  DMC  Huntsville. 
This  Megacenter  has  a  relatively  stable  environment  because  it  is 
not  scheduled  to  receive  additional  legacy  site  workload,  nor  has 
it  been  optimized.  The  prototype  will  attempt  to  develop  interim 
staffing  targets  based  on  measurable  workload  factors.  If  this 
effort  is  successful,  the  methodology  may  be  applied  to  other 
Megacenters  which  are  stable  but  not  yet  optimized. 

2.  Our  detailed  management  comments  to  the  draft  report  are 
enclosed.  The  point  of  contact  for  this  action  is  Ms.  Sandra 
Leicht,  Audit  Liaison.  If  you  have  any  questions,  Ms.  Leicht  can 
be  reached  on  (703)  607-6316. 

FOR  THE  DIRECTOR: 


1  Enclosure  a/s 


Copy  to: 

WESTHEM  RIG 

Quality  Information  for  a  Strong  Defense 


Defense  Information  Systems  Agency  Comments 


MANAGEMENT  COMMENTS  ON 
STAFFING  REQUIREMENTS  FOR  THE 
DEFENSE  MEGACENTERS  (4RE- 5034 . 00 ) 

1.  FINDING:  The  draft  report  states:  "Using  millions  of 
instructions  per  second  (MIPS)  as  the  workload  factor  in 
determining  the  total  staffing  requirements  of  3,073' for  the 
Defense  Megacenters  is  inappropriate." 

RESPONSE:  Nonconcur  with  the  finding.  The  draft  report 

states  that  using  MIPS  as  the  workload  factor  in  determining  the 
total  Megacenter  staffing  requirement  is  inappropriate  because 
MIPS  is  an  expression  of  how  fast  the  computer  central  processing 
unit  can  process  instructions.  The  draft  report  goes  on  to  cite 
the  textbook  definition  of  MIPS  as  the  number  of  instructions  the 
central  processing  unit  (CPU)  of  a  computer  can  process  in  a 
second.  While  this  definition  is  correct,  it  is  not  complete. 

It  is  common  practice  in  the  industry  to  use  MIPS  as  a  measure  of 
data  processing  workload,  as  well  as  processing  speed.  Strictly 
speaking,  workload  should  be  expressed  as  "millions  of 
instructions  (MI)"  leaving  "millions  of  instructions  per  second 
(MIPS)"  as  measure  of  processor  speed.  However,  the  grammatical 
subtleties  tend  to  get  overlooked  when  MIPS  is  used  to  indicate 
the  size  (in  terms  of  processing  capacity)  of  a  data  center,  or 
the  expected  processing  requirement  of  a  new  application. 

The  draft  report  claims  that  MIPS  has  no  relationship  to  workload 
functions,  such  as  computer  tape  changes,  console  commands,  input 
and  output  functions,  or  paper  changes,  performed  by  computer 
personnel .  This  conclusion  is  contrary  to  that  of  KPMG  Peat 
Marwick  as  documented  in  their  report  "Best  Data  Center 
Practices"  developed  for  DISA  in  September  1993.  The  research 
carried  out  by  Peat  Marwick  using  its  benchmarking  data  base,  and 
the  resulting  report  which  provided  a  detailed,  quantitative 
description  of  the  best  practices  of  data  centers  in  the 
commercial  environment  played  an  important  role  in  the 
development  of  DMRD  918  and  the  entire  DoD  data  center 
consolidation. 

In  describing  the  benchmarking  process  Peat  Marwick  used  to 
evaluate  data  center  performance  they  write: 

"Critical  to  our  technique  is  our  measurement  of  the  size 
of  the  data  center.  We  measure  size  in  millions  of 
instruction  per  second  (MIPS)  used  by  the  data  center 
(also  known  as  overall  CPU  utilization)  regardless  of  the 
MIPS  installed  in  the  data  center.  We  measure  data 
center  size  as  a  function  of  used  MIPS  since  we  are 
attempting  to  measure  the  size  of  the  workload  (emphasis 
added)  ,  not  the  capacity  of  the  data  center.  We  have 
correlated  used  MIPS  to  installed  MIPS  in  the  data 
centers  we  have  evaluated,  and  this  shows  a  strong 
correlation.  . 
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Not  only  does  Peat  Marwick  employ  used  MIPS  as  a  measure  of  data 
center  workload,  they  have  also  carried  out  the  research 
necessary  to  establish  the  correlation  between  used  MIPS  and 
other  data  center  workload  functions.  For  example,  Peat  Marwick 
has  found  that  the  correlation  between  installed  DASD  and  used 
MIPS  is  .92.  This  means  that  DASD  management  functions  dependent 
on  the  quantity  of  DASD  tend  to  increase  as  the  used'  MIPS 
increase.  They  have  found  a  similar  direct  relationship  between 
the  number  of  tape  mounts  and  used  MIPS.  In  this  case  the 
correlation  is  .66,  less  than  that  for  DASD,  but  st^ll  a  strong 
relationship  between  MIPS  and  a  workload  function  performed  by 
data  center  personnel.  Peat  Marwick's  report  also  addressed  the 
relationship  between  print  volume  and  used  MIPS.  In  this  case 
they  provide  a  figure  showing  a  clear  relationship,  but  no 
correlation  coefficient.  Peat  Marwick  quantifies  their 
evaluation  of  a  data  center  in  a  series  of  key  benchmark  data 
elements.  One  prominent  category  of  this  series  is  "Staffing 
Based  Upon  Workload"  which  includes  the  following  empirical 
metrics : 


Key  Data  Element  Average  Site 
Total  Staff  Per  Used  MIPS  1.93 
Console  Operators  Per  Used  MIPS  0.13 
Systems  Programmers  Per  Used  MIPS  0.16 
Capacity  Planners  Per  Used  MIPS  0.03 


Best  Practice 
0.70 
0 . 06 
0.03 
0.01 


The  research  carried  out  by  Peat  Marwick  demonstrates 
conclusively  that  used  MIPS  is  an  appropriate  workload  measure 
for  estimating  mainframe  staffing  requirements.  It  may  not  be 
the  ideal  workload  measure {s),  but  in  the  absence  of  other 
measures  related  to  functions  performed  by  personnel,  it  can  be 
used  to  estimate  the  total  staffing  required  to  process  a 
specified  workload  measured  in  MIPS. 

In  addition  to  the  mathematical  basis  for  using  MIPS  to  estimate 
megacenter  staffing  requirements,  there  is  also  a  financial 
basis.  The  Defense  Megacenters  were  established  as  fee-for- 
service  operations  in  the  Defense  Business  Operating  Fund  (DBOF) . 
DISA  was  granted  the  right  of  first  refusal  for  DoD  data 
processing  requirements  for  two  years  after  capitalizing  the 
Megacenters.  After  the  two  year  window,  customers  are  free  to 
take  their  business  wherever  they  can  get  the  best  deal .  That 
means  the  Defense  Megacenters  will  shortly  be  in  direct 
competition  with  private  sector  service  centers  for  business.  To 
compete  successfully,  the  Defense  Megacenters  must  become  as 
efficient  as  their  commercial  competition.  Data  on  total 
staffing  requirements  per  used  MIPS  in  the  private  sector  is 
available  from  several  sources.  This  data  was  considered  in 
establishing  the  staffing  targets  for  Defense  Megacenters. 
Although  this  approach  does  not  explicitly  recognize  the 
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technical  concerns  outlined  in  the  draft  report  about  using  MIPS 
to  determine  staffing  needs,  it  does  recognize  that  the  Defense 
Megacenters  must  remain  in  business  or  the  appropriateness  of  the 
workload  measure  used  to  establish  staffing  targets  has  no 
meaning. 

Data  center  operating  costs  are  generally  split  almost  equally 
between  labor  and  non-labor.  The  non-labor  component  consists  of 
computer  hardware,  software,  supplies  and  facility  costs  which 
will  not  differ  significantly  between  a  Defense  Megacenter  and  a 
commercial  service  center.  If  we  cannot  gain  an  advantage  in 
non- labor  costs,  that  means  we  must  realize  the  same  level  of 
labor  efficiency  achieved  by  the  private  sector  in  order  to 
remain  competitive.  This  is  just  another  way  to  say  that  the 
average  staff  per  used  MIPS  across  all  the  Defense  Megacenters 
must  be  reasonably  close  to  the  level  realized  in  the  commercial 
environment. 

2 .  RECOMMENDATION  1 :  Use  measurable  workload  functions 
performed  by  computer  personnel  rather  than  the  speed  in  which 
the  computer  processes  an  instruction  (as  expressed  in  millions 
of  instructions  per  second)  in  determining  the  staffing 
requirements  for  the  16  Defense  Megacenters. 

RESPONSE:  Nonconcur  with  the  recommendation.  In  developing 

staffing  targets  for  Defense  Megacenters  in  the  optimized  end 
state,  we  face  the  challenge  of  estimating  how  many  people  will 
be  required  from  two  to  five  years  in  the  future  to  staff  an 
operation  which  does  not  exist  today.  The  draft  report  describes 
several  different  methodologies  using  measurable  workload 
functions  performed  by  personnel.  Each  of  these  methodologies, 
however,  requires  quantitative  historical  data  to  produce 
staffing  estimates.  We  have  been  unable  to  collect  historical 
data  for  an  operation  which  does  not  yet  exist.  Moreover,  we 
cannot  even  estimate  workload  measures  for  the  optimized 
environment  until  we  complete  a  comprehensive  Concept  of 
Operations  which  includes  detailed  information  on  the  optimized 
hardware  and  software  environment.  This  information  will  not  be 
available  at  the  earliest  until  the  end  of  FY  95.  In  the 
interim,  the  only  reasonably  accurate  metric  we  have  on  the 
future  end  state  environment  is  total  projected  workload  measured 
in  MIPS.  Given  there  is  no  other  reasonable  alternative,  we 
intend  to  continue  developing  end  state  staffing  targets  using 
projected  end  state  workload  in  MIPS. 

3.  RECOMMENDATION  2:  Revise  the  staffing  requirements  for  the 
16  Defense  Megacenters  based  on  using  the  measurable  workload 
factors  discussed  in  Recommendation  1. 

RESPONSE:  Concur  in  part  with  the  recommendation.  As  noted 

in  the  response  to  Recommendation  1,  DISA  will  continue 
developing  end  state  staffing  targets  for  Defense  Megacenters 
using  projected  end  state  workload  in  MIPS  because  there  is  no 
other  viable  option.  It  is  possible,  however,  that  measurable 
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Final  Report 
Reference 


Revised 


workload  factors  may  be  used  to  establish  interim  staffing 
targets  during  the  migration  period  before  the  Megacenters  are 
optimized.  DISA  is  now  at  a  point  where  the  routine,  systematic 
review  of  our  total  organization  can  be  accomplished,  using  tools 
which  measure  workload  volume,  accomplishment  times,  and 
personnel  unit  cost  per  output.  These  reviews  are  scheduled  to 
begin  in  FY  1995  with  a  prototype  effort  at  DMC  Huntsville.  This 
Megacenter  has  a  relatively  stable  environment  because  it  is  not 
scheduled  to  receive  additional  legacy  site  workload,  nor  has  it 
The  prototype  will  attempt  to  develop  interim 
starring  targets  based  on  measurable  workload  factors*.  If  this 
effort  is  successful,  the  methodology  may  be  applied  to  other 
Megacenters  which  are  stable  but  not  yet  optimized, 

4.  RECOMMENDATION  3:  Disclose  in  the  cost  analysis  that 
projected  staffing  requirements  will  be  revised  using  measurable 
workload  factors  if  staffing  requirements  are  not  corrected 
before  the  cost  analysis  is  presented  to  Congress  by  March  1995. 

RESPONSE:  Nonconcur  with  the  recommendation.  DISA  will  use 
the  most  accurate  staffing  targets  available  at  the  time  the  cost 
analysis  is  presented  to  Congress  in  March  1995.  MIPs  is  a 
measurable  factor,  the  most  reasonable  tool  to  use  at  this  time 
and  does  not  require  correction.  Since  the  additional 
"measurable  workload  factors"  addressed  in  the  Draft  report 
cannot  be  used  to  project  staffing  requirements  until  FY  96  at 
the  earliest,  it  is  not  appropriate  to  raise  this  issue  in  the 
March  1995  report  to  Congress.  The  DISA  Comptroller  is  the 
functional  proponent  for  the  Efficiency  review,  Position 
Management,  and  resource  Requirements  Determination  process.  By 
1  March  95,  the  DISA  Comptroller  will  develop  an  E  schedule ' that 
will  insure  that  a  routine,  systematic  review  of  DISA' s  total 
organization  is . accomplished,  to  include  the  development  of  Most 
Efficient  Organizations  (MAO)  and  staffing  standards  for  both 
DBOF  and  appropriated  fund  activities. 
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